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This  paper  deals  with  the  application  of  operations  research 
and  systems  analysis  techniques  to  a  test  methodology  study 
(CCMBATEST)  being  conducted  by  the  Infantry  Board/-- a  subordinate 
testing  agency  of  the  US  Army  Test  and  Evaluation  Command,  Aberdeen 
Proving  Ground,  Maryland. 

The  mission  of  the  Infantry  Board  is  to  test  and  evaluate 
military  equipment  for  the  Armv--vith  emphasis  on  that  equipment 
which  is  designed  for  the  infantryman.  Since  the  Board  tests  equip¬ 
ment  which  will  be  used  in  combat,  it  conducts  tests  under  conditions 
representing  as  nearly  as  possible,  those  combat  conditions  soldiers 
may  expect  to  find  on  the  battlefiel^.-s 

In  1964,  the  Commanding  General  of  Test  and  Evaluation 
Command  wrote  a  letter  to  the  President,  Infantry  Board,  in  which  he 
commented,  "I  note  that  your  tests  are  conducted  under  simulated 
combat  conditions.  It  appears  to  me  that  you  are  using  too  much 
simulation  and  too  little  combat.  Please  conduct  a  study  to  Improve 
the  realism  of  testing  in  a  combat  environment,  and  to  improve  test¬ 
ing  methodology."  As  a  recult  of  the  study  Initiated  by  this  cor¬ 
respondence,  the  Infantry  Board  was  later  directed  by  Headquarters, 
Test  and  Evaluation  Command,  to:  (1)  determine  those  factors  which 
are  critical  to  the  evaluation  of  Infantry  weapons  in  a  quasl- 
tactical  environment,  and  (2)  develop  techniques  and  methods  for 
generating  meaningful  numerical  measurements  of  critical  factors  on 
a  real-time  basis.  In  support  of  these  objectives,  the  Board  is 
conducting  a  5-year  program  of  experimentation,  which  has  as  its 
immediate  purpose:  to  determine  those  test  factors  which  must  be 
emphasized;  to  produce  meaningful  performance  measurements  withf^ie  f"*, 
smallest  possible  expenditure  of  resources;  and  to  determine  those 
test  factors  which  may  be  de-emphasized  or  ignored.  By  determining 
these,  the  Infantry  Board  hopes  to  be  able  to  reduce  sampld  size, 
and  to  reduce  testing  time,  while  still  producing  significant  Q  ] 
results  in  the  evaluation  of  the  man/weapon  system.  J  |[  ... 
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The  accomplishment  of  the  program's  purpose  is  complicated 
by  the  fact  that  each  individual --the  gunner,  the  operator,  the 
soldier — becomes  an  integral  part  of  the  weapon  system,  and  there¬ 
fore  neither  the  weapon  nor  the  operator  can  be  isolated;  neither 
can  the  man/weapon  system  be  isolated  from  its  environment.  The 
critical  test  factors  being  sought,  then,  can  be  found  only  by  a 
total  systems  evaluation  of  the  man  and  his  weapon,  in  a  combat 
environment . 

Since  systems  research  and  evaluation  is  an  iterative  pro¬ 
cess  that  combines  many  activities  in  a  series  of  steps  or  stages, 
the  Board's  process  is  one  of  establishing  requirements,  test 
design,  experimentation  and  redesign.  It  is  a  continuous  activity, 
requiring  a  continuing  search  for  answers  to  the  following  questions: 

a.  What  criteria  are  most  appropriate  for  evaluation 
of  each  portion  of  the  system? 

b.  How  do  the  system  variables/factors  relate  to  the 

criteria? 


c.  Of  the  design  components  that  seem  possible  and 
feasible,  which  factors  represent  the  greatest  potential  improvements 
to  the  entire  system? 

The  criterion  most  appropriate  for  evaluation  of  the  man/ 
weapon  system  is  effectiveness  of  fire;  this  can  be  defined  as  the 
spatial  and  temporal  distribution  of  hits  over  a  specified  target 
area.  This  area  may  include  personnel  and  vehicular  targets,  or  it 
may  simply  be  a  piece  of  real  estate.  The  dependent  variables 
applicable  to  this  criterion  are  time  to  first  hit,  number  of  rounds 
necessary  to  achieve  first  hit,  total  hits  as  a  function  of  time, 
and  total  time  of  engagement.  In  addition,  suppressive  fire  must 
be  measured,  because  the  ability  to  achieve  a  near  miss  which  would 
suppress  the  actions  of  the  enemy  of  ter.  becomes  of  extreme  impor¬ 
tance  in  the  evaluation  of  effectiveness  of  fire.  Consequently, 
the  dependent  variables  also  include  time  to  first  near  miss,  and 
patterns  of  near  misses  as  a  function  of  time.  Other  measurements 
which  become  important  in  an  evaluation  of  the  man/weapon  system 
are  time  to  activate  weapon,  rate  of  fire,  ammunition  load,  marks-p 
manship,  and  individual  physical  characteristics.  The  importance  * 
of  these  as  evaluators  of  man/weapon  performance  varies  with  the 
characteristics  of  the  environment,  which  include  such  things  as 
the  tactical  situation,  weather,  climate,  vegetation,  and  soil 
composition. 

The  Board  is  working  with  the  smallest  unit  on  the  battle¬ 
field — the  rifleman.  This  has  two  implications:  first  because  of 
the  density  of  the  rifleman  in  combat,  very  small  improvements  in 
the  man/weapon  system  may  mean  very  large  gains  in  battlefield 
effectiveness.  An  incremental  improvement  of  one  in  the  4th  decimal 
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place  is  not  a  significant  improvement  for  a  single  rifleman;  but 
multiplied  across  three  divisions,  it  may  be  extremely  influential  in 
terms  of  enemy  attrition,  or  our  own  soldier  survivability.  The 
second  implication  is  obvious,  then.  How  does  one  measure  very  small 
differences?  The  answer  lies  in  sensitivity,  and  resolution.  The 
tools  provide  the  needed  sensitivity;  the  experimental  design  re¬ 
solves  measured  outputs  in  terms  of  the  design  variables. 

The  necessary  tools  to  undertake  such  evaluations  were  not 
available  when  the  Infantry  Board  began  this  study.  As  often  occurs 
in  other  undertakings,  before  a  task  can  be  initiated,  the  tools  to 
do  it  must  be  designed.  This  is  the  stage  of  the  current  effort. 

The  Board  is  in  the  process  of  designing  instruments  sensitive  enough 
to  isolate,  and  measure,  those  factors  which  significantly  influence 
the  effectiveness  of  the  man/weapon  syptem.  The  instruments  which 
provide  informational  inputs  during  current  firing  exercises  on  the 
range  include  an  FM-telemetered  round -count  device  with  each  firer, 
hardwired  shock  transducers  or  photoelectric  cells  to  locate  the 
position  of  each  firer  in  meters  from  the  target,  hit-sensitive  tar¬ 
gets,  and  near  miss  sensors.  The  time  resolution  of  these  different 
sensors  permits  the  measurement  of  the  performance  of  each  firer, 
whether  alone  or  in  a  group.  The  FM  signal,  and  the  position  loca¬ 
tion  signal,  locate  the  point  of  origin  of  the  round.  Knowing  the 
point  of  origin  of  the  round,  the  time  of  firing  the  round,  and  the 
time  at  which  the  round  passes  the  target  area,  enables  identifica¬ 
tion,  within  limits,  of  the  effect  of  each  specific  round  fired. 

Targets  are  laminated  plastic  foam  and  aluminum  (layers  of 
plastic--alumlnum — plastic --aluminum — plastic)  such  that  the  projec¬ 
tile  short  circuits  the  two  aluminum  foils  when  the  target  is  hit. 

The  short  circuit  is  cleared  about  a  hundred -thousandth  of  a  second 
later  as  the  bullet  completes  passage  through  the  target.  The  pulses 
generated  by  the  short  circuits  are  recorded  at  an  instrumentation 
center.  A  hit  on  target  also  causes  the  target  to  fall,  thereby  pro¬ 
viding  informational  feedback  to  the  firer  which  simulates  battle 
conditions,  and  enables  him  to  adjust  his  sight  picture  or  to  modify 
his  range  estimation.  Although  the  recording  system  observes  very 
few  Instances  of  simultaneous  firing,  squad  members  firing  on  the 
range  frequently  do.  If  two  individuals  fire  on  the  same  target 
within  a  second,  and  one  of  them  scores  a  hit,  both  assume  they  have 
scored  hits,  and  both  will  shift  fire  to  another  target.  This,  too, 
adds  to  the  realism  of  the  battlefield  simulation.  A  target,  when 
hit,  will  fall,  and  may  be  made  to  rise  within  the  variable  cycle 
time  of  2  to  4  seconds. 

Location  of  misses  in  the  target  area  is  now  determined  by 
microphones  which  record  the  amplitude  of  the  shock  wave  of  each 
passing  round.  But  the  system  is  being  converted  to  one  which  will 
locate  such  misses  mathematically.  Miss  distances  will  be  calcu¬ 
lated  on  the  basis  of  time  differences  of  the  arrival  of  the  shock 
wave  at  adjacent  transducers. 
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Shock  wave  transducers  for  miss  detection  are  mounted  di¬ 
rectly  behind  the  targets.  A  miss  will  be  heard  by  the  transducer 
behind  the  target  for  which  it  had  been  aimed,  and  on  the  adjacent 
target  transducers  as  well. 

The  formula  used  to  locate  misses  mathematically  is  repro¬ 
duced  below.  If  Dq  represents  the  distance  from  the  bullet  path  to 
the  target  transducer  at  which  the  round  had  been  aimed,  D.  the  dis¬ 
tance  to  the  first  transducer  to  the  left,  and  D£  the  distance  to 
the  first  transducer  to  the  right,  then  the  general  form  of  the  equa 
tion  is  derived  from  the  distance  formula: 

D0  -  D]  -  ~][  (X  -  X0)2  +  (Y  -  Y0)2  -  ij  (X  -  XL)2  +  (Y  -  Y^2 

Y2  -  AX2  -  BX  +  C 

A  second  equation  is  produced  by  the  time  difference  Dq  -  D£.  These 
equations  are  hyperbolic,  and,  when  solved  simultaneously,  will  pro¬ 
duce  four  possible  values  of  X  and  Y.  By  judicious  selection  of 
origins  (X  0,  the  leftmost  transducer;  Y  =  0,  the  ground  level), 
all  negative  values  of  X  and  Y  are  meaningless,  and  only  the  numeri¬ 
cal  values  in  the  first  quadrant  are  significant.  This  sometimes 
provides  two  values  of  X  and  Y,  but  they  differ  by  orders  of  magni¬ 
tude,  and  the  correct  set  of  coordinates  is  readily  discernible. 

At  the  same  time  that  the  Infantry  Board  is  designing  ie 
tools  to  measure  sensitive  factors,  it  is  also  in  the  process  of 
developing  experimental  designs.  Since  nol  enough  data  have  been 
accumulated  on  the  effects  of  such  things  as  range  distances,  firing 
positions,  target  configurations,  or  scoring  techniques,  which  are 
the  most  essential  and  critical  parameters  of  comparative  studies, 
it  is  necessary  to  Investigate  these  areas  systematically,  to  deter¬ 
mine  those  experimental  variables  which  should  form  the  basis  of 
small  arms  tests.  All  of  them  patently  cannot  be  examined  in  a  sin¬ 
gle  study,  but  in  a  series  of  consecutive  studies  it  should  be  pos¬ 
sible  to  identify  the  critical  variables.  For  example,  one  firing 
position  may  be  more  critical  than  all  other  firing  positions  when 
weapons  must  be  evaluated,  or  a  certain  range  distance,  or  distances 
must  inevitably  be  Included  whenever  one  wishes  to  conduct  a  com¬ 
parative  evaluation  of  weapons.  Success  in  the  search  for  those 
factors  which  must  be  emphasized,  and  those  which  may  be  e'e- 
emphasized,  depends  heavily  on  the  ability  to  develop  designs  end 
instrumentation  to  gather  necessary  data  under  meaningful  conditions 

Since  the  estab1 ishment  of  true  mathematical  relationships 
among  the  variables  of  a  combat  environment  is  still  far  in  the 
future,  simulation  models  are  playing  an  important  role  in  drawing 
conclusions  from  measurable  situations.  The  model  provides  the 
first  estimate  of  the  mathematical  relationship  between  factors. 
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What  are  some  of  the  influencing  factors?  First,  the  man  part  of  the 
man/weapon  system — the  human  factors.  To  what  degree  do  physical 
attributes  relate  to  combat  effectiveness?  Marksmanship?  Fatigue 
levels?  Ability  to  estimate  range?  Next,,  weapon  differences.  What 
comparative  differences  are  inherent  in  recoil,  sights,  trajectory, 
rate  of  fire,  mode  of  fire,  ammunition  supply?  Then  come  environ¬ 
mental  effects:  rain,  cold,  heat,  mud,  dust.  Furthermore,  the 
rifleman  rarely  goes  it  alone.  How  does  the  individual  squad  member 
react  at  a  specific  time  to  a  given  set  of  stimuli?  Which  controls, 
weapon  mixes,  weapon  numbers  most  affect  individual  man/weapon  per¬ 
formance?  How  does  feedback  from  rounds  fired  on  the  objective  by 
other  squad  members  affect  individual  performance?  Additionally, 
what  are  the  tradeoffs  between  firepower  and  accuracy,  firepower  and 
the  weight  to  be  carried  by  the  individual  soldier?  The  first  prob¬ 
lem  being  attacked  therefore  is  to  ascertain  which  are  the  major 
factors  that  influence  man/weapon  system  performance.  This  requires 
quantification  of  the  combat  environment --a  massive  undertaking,  and 
one  which  is  only  beginning  to  be  developed.  But  this  is  the  path 
the  Board  will  follow  in  seeking  answers  to  the  questions  above. 

Ranges  that  will  provide  the  basic  building  blocks  for 
analysis  are  now  in  the  process  of  being  built.  A  small  arms  attack 
range  is  already  operational;  a  quick-fire  range  has  been  completed 
and  is  undergoing  testing.  Three  other  ranges — a  small  arms  defense 
range,  an  indirect -fire  range,  and  an  antitank  range --will  be  con¬ 
structed  during  the  next  2-year  period.  Each  range  has,  or  will 
have,  instrumentation  which  records  the  performance  as  a  function  of 
time,  of  each  round  fired,  each  hit,  and  each  near  miss. 

1 

There  follows  a  discussion  of  some  of  the  results  of  the 
first  test  on  the  attack  range — the  first  range  completed — to  demon¬ 
strate  the  quantity  of  data  and  the  degree  of  resolution  thus  far 
achieved.  These  output  data  will  become  the  criteria  for  evaluating 
the  design  of  later  ranges,  Instrumentation,  and  experimental  design, 
and  will  ultimately  be  used  as  input  to  command,  control,  and  organi¬ 
zational  studies.  Analysis  of  these  results  has  demonstrated  that 
the  ranges  can  measure  significant  variables  in  the  performance  of 
the  man/weapon  system.  The  results  of  the  first  test  have  been  em¬ 
bodied  in  a  report  dated  April  1966,  entitled.  Pilot  Experiment, 
Attack  Experiment  I,  Small  Arms  Service  Test  Design  for  a  Study  of 
Small  Arms  Service  Test  Facilities  and  Methods.  It  is  emphasized 
that  this  experiment  was  condacted  to  test  the  range--not  to  compare 
the  weapons. 

There  are  two  ways  of  improving  test  methodology:  increase 
the  quantity  of  data,  and  increase  the  quality  of  data.  Figure  1 
shows  the  quantitative  increases  experienced  during  the  first  field 
experiment.  The  primary  measure  of  performance  on  firing  ranges  in 
the  past  has  been  in  terms  of  the  percentage  of  hits  achieved,  under 
specified  conditions,  measured  at  the  conclusion  of  an  exercise. 

Under  such  circumstances,  when  firing  at  an  attack  objective  from  150 
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meters,  about  12%  of  the  available  data  could  have  been  measured. 
This  means  that  nothing  would  have  been  known  about  88%  of  the  rounds 
fired — except  that  they  had  been  fired.  By  adding  near  miss  and 
short  round  sensors,  measurable  output  data  were  increased  to  97%. 

The  quality  of  the  data  is  empirically  demonstrated  in  the  figures  2 
through  6.  Figure  2  shows  the  target  hits,  by  range,  for  both  weap¬ 
ons.  These  data  showed  rifle  A  superior  in  hit  probability  at  all 
distances  on  the  range,  and  superior  by  a  statistically  significant 
difference  at  6  out  of  17  of  the  distance  increments.  Comparison  of 
reacquire  times  (figure  3)  showed  little  difference  between  the  fir¬ 
ing  positions  tested  (supported  kneeling  and  prone) ,  but  did  uncover 
a  significant  difference  between  the  two  weapons.  Other  data  sup¬ 
ported  the  conclusion  that  firers  encountered  more  difficulty  in 
rapid  acquisition  of  a  steady  sight  picture  with  rifle  B. 

Figure  4  presents  an  analysis  of  the  time  necessary  to 
change  magazines.  Here,  a  small  difference,  not  considered  signifi¬ 
cant,  in  favor  of  rifle  B  was  found.  In  computing  the  data  shown  on 
figure  4,  pure  magazine  change  time  was  identified  by  subtracting 
reacquire  time  from  the  reloading  manipulation.  It  was  found  that 
the  time  required  to  reload  and  to  reacquire  the  target  averaged 
approximately  15  seconds.  The  Infantry  Board  considered  this  exces¬ 
sive,  and  this  experiment  became  the  basis  for  a  Board  recommendation 
to  higher  headquarters  that  there  be  an  urgent  improvement  of  the 
magazine  component  of  current  small  arms  weapons  systems. 

Figure  5  shows  a  sample  of  information  produced  by  combin¬ 
ing  weapons  performance  in  terms  of  near  misses  in  the  two  basic 
firing  positions.  The  curves  Indicate  superiority  of  the  prone  posi¬ 
tion  at  longer  ranges,  but  oddly  enough  favor  the  kneeling  position 
at  closer  ranges,  possibly  due  to  improved  visibility  of  the  higher 
head  position. 

Assault  fire  information  (figure  6)  displayed  for  the  88- 
meter  to  40-meter  range  firings,  compared  to  hits  per  rounds  fired 
from  150  meters  to  88  meters  (in  the  kneeling  and  prone  positions), 
indicates  the  degradation  of  performance  during  all  trials,  by  all 
players,  as  they  initiated  assault  firing.  The  percentage  of  hits 
in  the  assault  is  only  5%  or  lower.  Performance  in  the  assault  is 
significantly  different  from  performance  in  the  kneeling  and  prone 
positions  with  both  rifles,  and  it  is  clear  that  the  technique  of 
using  assault  fire  deserves  further  analysis. 

In  summary,  as  can  be  seen  from  the  charts,  the  data  base 
already  accumulated  in  the  field  experiment  permits  display  of  com¬ 
parative  curves  of  weapons  performance,  singly  or  in  .-.ombination,  as 
affected  by  any  of  the  independent  variables  chosen  in  the  experi¬ 
mental  design. 

These  are  the  building  blocks  with  which  we  move  to  the 
next  level  of  penetration.  Each  new  level  achieved  in  testing 
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procedures  provides  the  stepping  stone  to  higher  and  more  sophisti¬ 
cated  plateaus.  The  CCMBATEST  concept  is  an  iterative  process  which 
has  significantly  advanced  test  methodology,  and  from  which  has 
evolved  development  of  test  facilities  which  permit  more  rapid,  and 
more  accurate,  evaluation  of  the  man/weapon  system. 

The  Infantry  Board  is  pleased  with  the  progress  already 
made  in  development  of  the  CCMBATEST  concept,  and  the  concept  has 
been  integrated  into  Board  operations,  supporting  our  motto--ONLY 
THE  BEST  FOR  THE  FINEST. 
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FIGURE  4 
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